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D2D  Consortium  Goals 


JASON  recommendation:  Create  consortium  focused  on 
each  challenge  problem,  where  success  is  defined  at  the 
consortium  level,  not  at  the  individual  contractor  level 

Consortium  should  be  an  integrated  team  conducting 
translational  research 


Consortium  addresses  the  challenge 
problems  in  a  coherent  and  integrated 
team  approach  \ 
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Consortium  openly  shares  and 
documents  modules  within  the 
consortium 
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Scientists 


Shared 
Scientific  Agenda 


New  modifications  to  modules  are 
evaluated  and  managed  by  the 
consortium 
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Consortium  Challenges 


Consortium  should  be  an  integrated  team 
conducting  translational  research 


■  BAA  response  may: 

Be  skewed  towards  parts  of  the  problem/challenge 
Have  a  mix  of  complete  and  partial  solutions 
Have  diversity  in  performer  capability 

Consortium  addresses  the  challenge  problems  in 
coherent  and  integrated  team  approach 


■  Team  dynamics  complicated  by  performance 
diversity  (geo/temporal,  organizational 
culture/capability,  etc.)  and  individual 
incentives^gendas 

■  Proprietary  approaches  may  need  integration 
with  non-proprietary  solutions  in  build 

This  is  the  reality  of  the  field 
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Consortium  Challenges 


Consortium  openly  shares  and  documents  ^ 
modules  within  the  consortium 


■  What  will  be  the  incentive  to  share  as  ^  - 

much  or  as  genuinely  as  possible?  s 

i 

New  modifications  to  modules  are  evaluated  and 


managed  by  the  consortium 

■  This  implies  consensus  across  the 
consortium  and  may  present  threats 
to  new  performers  over  time 
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FYll  Broad  Opportunity 


Opportunity  to  take  advantage  of 
testbed  "process"  information 

■  Early  ad  hoc  consortium  support 
mechanisms  and  processes  will 
establish  practices  that  set  precedent 
for  FY12+ 

■  Changes  to  program  operations  will  be 
more  difficult  to  implement  without 
the  necessary  methods  and 
infrastructure  in  place  to  control  and 
manipulate  the  testbed  and  spiral 
development  process  over  time 

Some  research  is  necessary  to  determine 
appropriate  architecture,  methods  and 
metrics  for  FY12+ 
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What  Can  be  Done  in  FYll  to  Support 
D2D  Consortium  Infrastructure  Elements? 


FYll  represents  an  opportunity  to 
evaluate  mechanisms  to  enable 
the  defined  D2D  Program 
Organization  (Spiral  Development) 

■  Tri-service  Data-to-Decisions 
Collaborative  Working  Group  (D2D-CWG) 

Establish  requirements  and  practices  for  consortium 
support 

■  Testbed  (FYll  variant,  FY12  final) 

Evaluation  and  development  of  test  harnesses  and 
gatekeeping  infrastructure  for  access 


BuUd 
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What  Can  be  Done  in  FYll  to  Support 
^  D2D  Consortium  Infrastructure  Elements? 


Data  Analysis  Center  (FY12)  y  BuUd 

■  Development  of  metrics  management  prowem  ^ 

infrastructure  and  innovative  ^  iLvXpV'iS 

multidimensional  visualization  r  '  r- 

techniques 

■  Investigate  AIR  WG  and  other 
prior-work/techniques  for  challenge 

problem  development,  metrics,  and  ongoing  management 

Data  Coordination  Center  (FY12) 

■  Develop  solutions  for  administrative  management  (core 
services)  and  their  impact  on  extended  core  services  and 
analytics  (R.  Avent  Data  Management  Layer  SOA) 

■  Assess  Data  management  layer  reliability/sustainability 
requirements 
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Near  Term 

(FYll  &  Early  FY12)  Strategy 


Capabilities  to  service  enabling  solutions 

■  View  the  BAA  products'  performance  data  as  data  to  be 
managed 

Performance  dashboards  and  visualizations 

Directory  service  for  descriptive  registration  of  performers' 
approaches 

■  Structure  the  end-to-end  testing  process  as  an  orchestration 

Algorithm  revision  control  and  annotation 

Develop  documented  testing  metrics  and  orchestration 

□  Formalize  challenge  problem  metrics  process 

□  Test  harnesses 

□  Open-source  test  suites  and  orchestrations 

Attention  to  integration  and  transition  to/from  "build"  (beyond 
evaluation) 

■  Develop  an  administrative  collaboration  workspace 

Example:  Discussion  blog/wlki/document  management 
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Long  Term  Vision  Ideas 


Exploit  the  D2D  Program  Organization  (Spiral 
Development)  and  testbed  meta  process  information  space 

■  Capture  intra-algorithm/module  metrics,  share  trends  and 
identify  gaps 

Develop  technical  transition  metrics 

□  Technical  transition  metrics:  ideas  of  semantic  consistency,  tri-service 
interoperability,  data  format  supportability,  etc. 

Use  information  from  the  meta  process  to  drive  performer 
motivations  (collaboration,  disclosure,  FY12+  consortium  policy) 

■  Provide  a  quantitative  framework  to  manipulate  FY  12+ 
consortium  management  functions  and  program/testbed 
CONORS 

Adapt  and  extend  prior  work  from  ATR  WG  (Westerkamp  et  al. 

2002;  Ashby,  Blasch,  et  al.  2002),  DISA  FDCE  (Federated 
Development  and  Certification  Environment),  and  others 
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